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Major Research Issue 


How to use artificial intelligence in system 

control? 


Replace human operator 


or 


Amplify human operator's abilities to monitor 
the system and detect, diagnose and 
compensate for system failures? 



Objective of OFMspert Research 


Design an architecture to provide the human 
operator with an intelligent decision 
support system 


- to augment, not replace, the versatile 
human skills with skills provided by 
machine intelligence. 

- to compensate for known human 
limitations. 

~ to complement individual human 
preferences 


Develop a theory of human-computer 

interaction in the control of complex, 
dynamic systems (normative, plausible) 


Build a model of the theory to demonstrate 
and empirically evaluate the proposed 
architecture (operator's associate) 



Requirements for an 
Intelligent Operator's Associate 


Operator's Associate must provide 

information and control abilities at 
the right time, of the right kind and 
with the ease of a human associate 

~ understanding 

— control 

— interface 


Understanding requires a model of the 

operator and system that can allow the OA 
to infer the operator's current 
control goals given knowledge of the 
control task, system functions, and current 
control state. 


OFMspert Characteristics 


OFMspert (operator Function Model Expert 
System) is an intelligent operator's 
associate based on the operator function 
model (Mitchell, 1987) 


OFMspert uses the Operator Function Model 
(OFM) to represent knowledge about the 
operator 


OFMspert uses the blackboard model of 

problem solving (Nil, 1986) to maintain a 
dynamic representation of operator goals, 
plans, tasks, and actions given previous 
operator actions and current system state 



Figure 1. A Generic Operator Function Model 








Figure 2. flCTIN's Architecture 


















COMPONENTS OF THE BLACKBOARD MODEL: 

Blackboard data structure 

Contains the set of partial and complete solutions known as 
the solution space. 


Divided into levels of information where each level 

represents a distinct level of abstraction in the solution 
space. 
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Blackboard data structure 









Experimental Validation 
of OFMspert's Intent Inferencing 


- Compare a domain expert’s interpretations 
of operator actions to OFMspert's 
interpretation of those actions. 


- Compare verbai protocols of subjects 
verbalizing their intentions for each 
action to OFMspert’s interpretations 


of those actions. 


Table 3. Experiment 1: Average Percentage of Equivalent 
Interpretations between ACTIN and a Human Domain Expert. 

Ordered by rank. 
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100% 

Endpoint telemetry page requests 

100 

Deconfigure 
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Telemetry page requests 

96.3 

Answer 

91.4 

Reconfigure 

91.2 

Interior telemetry page requests 

87.1 

Replace 

75.3 

Mission schedule page requests 

66.7 

MSOCC schedule page requests 

50.3 

Equipment schedule page requests 

21.8 

Events page request 

17.7 

Pending page request 

16.7 



Table 5. Experiment 2: Average Percentage of actions 

matched by OFMspert 


Configure 

Deconfigure 

Answer 

Replace 

Equipment schedule page requests 
Mission schedule page requests 
Interior telemetry page requests 
Endpoint telemetry page requests 
MSOCC schedule page requests 
Telemetry page requests 
Reconfigure 
Events page request 
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94.8 
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Pending page request 
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Table 2. Experiment 1: Proportions of Equivalent Interpretations 

between ACTIN and a Human Domain Expert 
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Answer 


Significantly good match 
# Significantly poor match 




Table 4. Experiment 2: Proportions of Equivalent Interpretations 

between ACTIN and Verbal Reports 


Telem 

Endpoint 

Telem 

Interior 

Telem 

MSOCC 

Sched 

Equip 

Sched 

Mission 

Sched 

Events 

Pending 

Deconfig 

Reconfig 
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Replace 

Answer 
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* Significantly good match (B> b(0.025,n,0.5) 

# Significantly poor match (B< n-b(0.025,n,0.5) 







Figure 10. Example of Ally's User Interface 
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Intelligent Tutoring System 
for Satellite Operators 
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Root Node: Supervisory Control of GT-MSOCC 
FI: Control of Current Missions 



Type of Failure 

Replaceable 

NonReplaceable 

Hardware Failure 

Sll 

S15 

No Data Relayed 

S12 

S16 

Half Normal Data 

S13 

S17 

Triple Normal Errors 

S14 

SIS 



F2: Configure to Meet Support Request 

F3: Compensate for Automated Schedule Failures 

F4: Manually Deconfigure a Mission 


Figure 3 A Task Model 








Instructional 

Module 



exercises, aiding, & performance feedback 



COMPLEXITY AND PROBLEM 
SOLVING 


Three basic elements in problem solving 
situations: 

The World to be acted on, 

The Agent who acts on the world, 

The external representation of the world 
utilized by the problem solving agent. 


AGENT 



WORLD 


WHAT MAKES 
PROBLEM SOLVING 
COMPLEX: 


DIMENSIONS OF COMPLEXITY 

• Dynamism 

• Number of parts and extensiveness of 
interconnections between parts 

• Uncertainty 

• Risk 

DOMAIN OF INTEREST: 

COMPLEX DYNAMIC SYSTEMS 

• Human operator as a supervisory controller 
• monitoring task 


• troubleshooting task 



QUESTIONS TO BE ADDRESSED 


What are the specific skills with respect 
to the four dimensions of complexity that 
are necessary to carry out the tasks 
involved in a CDS? 

What are the goals of an ITS designed for a 
CDS? What do we want the operator to 
learn? Are the goals attainable? 

What approaches in each module of an ITS 
seem appropriate to a CDS and why? How 
do we translate an approach in the context 
of a CDS? 

What about implementation issues and 
"do-ability"? How much of the CDS world 
should be represented in the ITS? 

How do we evaluate the ITS (if 
implemented) to test if the goals are 
attained? 




COMPONENTS OF AN 
INTELLIGENT TUTORIAL SYSTEM 


Domain Expertise 


Student Model 


Pedagogical Expertise 


Interface 


Figure 3. ACTIN's Intent Inferencing Structure 





REVIEW OF APPROACHES 


Domain Expertise: 

> Information-structure-oriented paradigm (SCHOLAR, 
1970) 

Hierarchical scripts (WHY, 1977) 

Finite state automata (METEOROLOGY, 1973) 

Multiple representations of procedural and 
declarative knowledge (SOPHIE I, 1975; RBT, 1986) 

Qualitative modelling (STEAMER, 1984) 

Probabilistic model (INTEGRATION, 1973) 

D-rules (MYCIN/GUIDON, 1979) 

Procedural networks (BUGGY, 1975) 

Generalized AND/OR graph (REPAIR theory, 1980) 

Problem-solving models: 

• Active structural networks (FLOW, 1974) 

• Linguistics theory (SPADE, 1976) 

• Dependency graphs (MACSYMA ADVISOR 1977) 

• Intention-based knowledge structure 
(PROUST, 1984) 

• Operator function model (AHAB, 1987) 



STUDENT MODEL 


• Differential model (WEST, 1976) 

• Overlay model (WUSOR-II, 1977; GUIDON, 
1979) 

• Buggy model (BUGGY, 1978; MENO-II; PROUST, 
1984) 

• Limited bug model (AHAB, 1987) 


INTERFACE 

• Textual (SCHOLAR, 1970; SOPHIE, 1975; WEST, 
1976; GUIDON, 1979; PROUST, 1984; etc) 


• Graphical (ALGEBRALAND, 1983; STEAMER, 
1984; IMTS, 1986; RBT, 1986; AHAB, 1987) 



REVIEW OF APPROACHES (cont'd) 


Pedagogical Expertise: 

Socratic method (WHY, 1977) 

Reactive learning environment (SOPHIE I, 

1975; MACSYMA ADVISOR, 1977) 

Conceptual fidelity (STEAMER, 1984; AHAB, 

1987) 

Progression of O-order qualitative models 
(QUEST, 1986) 

Curriculum Information Network (BIP, 1976) 

Exploratory learning (LOGO, 1980) 

Issues and examples paradigm (WEST, 1976) 

Increasingly complex microworlds paradigm 
(Fischer, et al., 1978) 

Expert-based coaching (WUSOR-I, 1976) 

Bite-sized architecture (SMITHTOWN, 1986) 

Layered curriculum and steering test concept (MHO, 
1987) 

Discourse management networks (MENO-TUTOR, 
1984) 

T-rules (GUIDON, 1979) 

ACT theory (GEOMETRY and LISP tutors, 1984) 



OFMTutor 


- Intelligent tutoring system for operators 
of complex dynamic systems 


- Based on the Operator Function Model (OFM) 















OFMTutor's Model of Expertise 

Model derived representative of 
Goals, Plans, and Tasks 


Goals 


Plans 


Tasks 


Actions 






OFMTutor's Student Model 

Data derived representation of 
goals, plans, and tasks 
based on student's actions 


Goals 


Plans 


Tasks 


Actions 





OFMTutor's Pedagogical Strategy 

and Diagnosis 

Guided discovery/coaching in context 
of system operation 

Differential modeling techniques that 
compare expert and student 
blackboard models 


Expert Blackboard Model 


Student Blackboard Model 


Goals 


Goals 


Plans 


Plans 


ft! 


Tasks 


Tasks 


Actions 


Actions 











Work in Process 


Design of a computer-based operator 
associate that evolves from tutor to 
assistant as the skills of the human 
operator change from novice to expert. 


The refinement of the Ally interaction 
to allow cooperative problem solving 
and repair of hypothesis formation. 


Evolution of a broader theory of 'good* 
architectures utilizing human and 
computer decision makers in 
interactive control. 


OFMTutor’s Interface 

Supports graphical, inspectable 
representation of joint hypotheses 
(expert and student) 

Model of discourse enables 
conversational capabilities 
and supports repair 
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Tasks 
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Actions 






